Computer-generated imagery typically involves using software and/or hardware to generate one or more images from a geometric model. A geometric model defines objects, light sources, and other elements of a virtual scene and a rendering system or other computer/software/hardware system will read in the geometric model and determine what colors are needed in what portions of the image. A renderer will generate a two-dimensional (“2D”) or three-dimensional (“3D”) array of pixel color values that collectively result in the desired image or images.
For a simple geometric model, such as a cube in a vacuum with a single light source, a simple computer program running on most computer hardware could render the corresponding image in a reasonable amount of time without much optimization effort. However, there are many needs—in the entertainment industry and beyond—for methods and apparatus that can efficiently process complex interactions of virtual objects to generate imagery in constrained timeframes where the images might need to convey realistic light interactions, such as light interacting with a participating medium.
Examples of such interaction include the scattering of light, which results in visual complexity, such as when the light interacts with participating media such as clouds, fog, and even air. Rendering this complex light transport typically involves solving a radiative transport equation [Chandrasekar 1960] combined with a rendering equation [Kajiya 1986] as a boundary condition. Other light interactions might include light passing through objects, such as refractive objects.
Rendering preferably involves computing unbiased, noise-free images. Unfortunately, the typical options to achieve this are variants of brute force path tracing [Kajiya 1986; Lafortune and Willems 1993; Veach and Guibas 1994; Lafortune and Willems 1996] and Metropolis light transport [Veach and Guibas 1997; Pauly et al. 2000], which are notoriously slow to converge to noise-free images despite recent advances [Raab et al. 2008; Yue et al. 2010]. This becomes particularly problematic when the scene contains so-called specular-diffuse-specular (“SDS”) subpaths or specular-media-specular (“SMS”) subpaths, which are actually quite common in physical scenes (e.g., illumination due to a light source inside a glass fixture). Unfortunately, path tracing methods cannot robustly handle these situations, especially in the presence of small light sources.
Methods based on volumetric photon mapping [Jensen and Christensen 1998] do not suffer from these problems. They can robustly handle SDS and SMS subpaths, and generally produce less noise. However, these methods suffer from bias. Bias can be eliminated, in theory, by using infinitely many photons, but in practice this is not feasible since tracking infinitely many photons requires unlimited memory.
Volumetric photon mapping is an approach to dealing with light in a participating medium. It is described in [Jensen and Christensen 1998] and subsequently improved by [Jarosz et al. 2008] to avoid costly and redundant density queries due to ray marching. In those approaches, a renderer formulates a “beam radiance estimate” that considers all photons along the length of a ray in one query. [Jarosz et al. 2011] showed how to apply the beam concept not just to the query operation but also to the photon data representation. In that approach, the entire photon path is used instead of just photon points, to obtain significant quality and performance improvement. This is similar in spirit to the concept of ray maps for surface illumination [Lastra et al. 2002; Havran et al. 2005; Herzog et al. 2007] as well as the recent line-space gathering technique [Sun et al. 2010]. These methods result in bias, which allows for more efficient simulation; however, when the majority of the illumination is due to caustics (which is often the case with realistic lighting fixtures or when there are specular surfaces), the photons are visualized directly and a large number is required to obtain high-quality results. Though these methods converge to an exact solution as the number of photons increases, obtaining a converged solution requires storing an infinite collection of photons, which is not feasible.
Progressive photon mapping [Hachisuka et al. 2008] alleviates this memory constraint. Instead of storing all photons needed to obtain a converged result, it updates progressive estimates of radiance at measurement points in the scene [Hachisuka et al. 2008; Hachisuka and Jensen 2009] or on the image plane [Knaus and Zwicker 2011]. Photons are traced and discarded progressively, and the radiance estimates are updated after each photon tracing pass in such a way that the approximation converges to the correct solution in the limit.
Previous progressive techniques have primarily focused on surface illumination, but [Knaus and Zwicker 2011] demonstrated results for traditional volumetric photon mapping [Jensen and Christensen 1998]. Unfortunately, volumetric photon mapping with photon points produces inferior results to photon beams [Jarosz et al. 2011], especially in the presence of complex specular interactions that benefit most from progressive estimation.
Thus, even with those techniques, there was room for improvement.